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SUMMARY 


Most  of  our  work  since  the  last  interim  report  was  spent  on  studying  necessary  adaptations  of  the 
software  to  various  operational  requirements,  assisted  by  data  nans  to  suggest  their  use,  and 
studying  and  assessing  recent  statistical  and  neural  theoretical  developments  for  possible 
application.  Main  activities  continued: 

requirements  analysis  for  development  of  prototype  into  a  system  for  operational  use; 
re-designing  the  basic  methodology  implemented  by  the  system,  involving  a  clear  separation 
of  the  main  methodological  elements; 

differentiating  project  pursuit  classification  and  pixel  blending  within  classes; 
conserving  inspection  mode  to  facilitate  selecting  (sub)  classes  and  regions  in  a 
classified  image  for  further  analysis; 

continue  study  of  spectral  de-mixing,  within  (sub)  classes,  using  RBF  neural  network 
technology. 
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Unsupervised  Spatial  Feature  and  Change  Detection  in  RS  Imaging 
Contract  no.  N68171-98-C-9012 
Fifth  Interim  Report 

(1)  Scientiflc  work  during  report  period 

Over  the  summer  we  received  signs  from  our  partners  in  Hanover  (NH)  that  our 
previous  prototype  had  aroused  much  interest  and  appreciation  during  demo’s  and 
tests  in  the  field.  Preparations  seem  to  have  been  made  to  integrate  or  incorporate  it  in 
the  operational  environment  of  some  USAGE  institutions.  Some  implementation 
difficulties  were  reported  to  us.  These  encouraging  experiences  supported  our 
conviction  that  this  system  of  unsupervised  classification  should  serve  as  a  highly 
efficient  and  discriminating  detection  tool  in  such  areas  as  water  control,  oil  spillage, 
environmental  monitoring,  land  usage,  and  change  detection,  where  multispectral 
remote  sensing  data  are  used.  Its  combination  of  an  innovative,  highly  effective 
method  of  adaptive,  targeted  pixel  sampling,  together  with  the  latest  efficient 
statistical  techniques  and  high  speed  algorithms,  should  make  it  a  fast  and  cutting 
edge  instrument  of  feature  detection  in  multispectral  RS-images,  compared  with 
current  legacy  methods. 

However,  although  this  prototype  might  well  be  used  as  a  supplementary  explorative 
aid  to  field  operations,  it  should  be  considered  unfit  for  inunediate  integration  in 
operational  processing  environments.  For  good  operational  use  some  additional 
features  and  data  management  possibilities  should  be  added,  which  necessitate  a 
further  re-engineering  of  the  system  beyond  the  capabilities  of  the  present  prototype. 

Consequently,  as  already  anticipated  in  the  fourth  interim  report,  we  shifted  our 
research  efforts  over  this  period  from  the  original  objectives  toward  a  redesign,  which 
would  incorporate  these  necessary  features  already  in  this  initial  stage  of  the  project. 
(In  the  original  proposal  this  stage  was  located  more  at  the  end  of  the  project,  so  as  to 
enable  the  incorporation  of  the  new  methods  which  are  the  main  target  of  the 
proposal). 

The  following  items  were  studied  and  prepared. 

1.  More  modular  redesign  of  the  system,  involving  a  clear  separation  of  the  main 
methodological  component; 

a)  adaptive  pixel  learning  sample  selection  options; 

-  homogeneous,  without  stratification; 

-  homogeneous,  with  stratification; 

-  biased  random,  with  stratification; 

-  local/global  density  selection. 

-  b)  multivariate  density  based  unsupervised  class  detection,  involving  class 

separation  tuning  for  optimal  detection; 

-  c)  the  various  image  classification  modes: 

-  1-NN  (nearest  neighbor)  mode; 

-  1*‘  PC  (principal  component)  projection  pursuit  mode; 
d)  the  various  blending  modes: 

-  no  blending; 

-  blending  along  f  ‘  PC  (principal  component)  axis; 
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Especially  the  introduction  of  all  combinations  in  the  simultaneous  application  of 
mode  c)  and  d),  which  was  not  possible  in  the  earlier  prototype,  will  be  necessary  in 
subsequent  applications.  For  instance,  different  line  segments  along  the  1®*  PC  should 
be  used  for  projection  pursuit  classification  on  the  density  classes  and  the  blending  of 
pixels  within  classes.  To  ensure  robust  and  reliable  classification,  a  relative  small 
segment  length  should  be  used  for  image  classification.  A  (much)  longer  segment 
should  be  used  to  represent  blending  of  pixels  within  a  class,  and  subsequent 
(de)mixing  analysis.  We  have  experimented  with  these  parameters  on  an  available 
data  set. 

2.  The  highly  innovative  pixel-by-pixel  classified  image  inspection  mode  should  give 
the  full  classified/blended  image,  together  with,  as  in  the  prototype,  per  pointed  pixel 
its  class  parameters  and  its  original  spectral  and  new  (class  blend)  values.  In  the 
earlier  prototype  this  possibility  is  volatile,  as  it  is  not  saved  when  exiting  a 
classification  run. 

For  operational  use  it  will  be  necessary  to  save  selected  classify/blended  images  in 
database  format  at  will,  in  order  to  re-inspect  or  re-analyze  the  image  at  a  later  stage: 

-  given  a  user  accepted  image  classification,  a  closer  pixel-by-pixel  investigation 
can  be  done  separately,  at  later  moments; 

-  the  underlying  classified  database  is  amenable  to  further  analysis,  using  other 
analysis  modules; 

-  the  image  classified/blended  database  (after  addition  of  geo-coordinates)  can  be 
used  in  the  context  of  GIS  spatial  statistical  techniques,  such  as  Kriging,  etc. 

3.  Moreover,  in  the  prototype  the  classified  GIF  image  and  Inspect  image  mode  are 
determined  by  initial  screen  size.  An  operational  version  should  enable  full  scrolling 
over  the  entire  *.bsq  image  size,  chosen  independent  of  screen  size. 

Some  additional  operational  features  are  essential  for  further  analysis  of  regions  and 
areas  in  terms  of  the  detected  classes  in  the  image. 

4.  For  instance,  given  a  classified  image: 

-  select  the  pixels  in  a  specified  class  for  further  analysis. 

-  select  a  contoured  region  (e.g.  a  known  polluted  or  oil  spit  spot)  within  the  classified 
image  to  focus  further  analysis  on  this  selection  ( i.e.  generalize  this  by  means  of 
spectral  or  class  vector  matching  across  the  image). 

We  have  continued  investigating  these  options,  as  they  are  necessary  for  the 
continuation  of  our  project  according  to  our  proposal. 

5.  We  have  continued  investigating  spectral  de-mixing,  within  (sub)classes,  to  be 
used  within  selections  as  mentioned  ad  4.  Spectral  demixing  is  essential  for  targeted 
oil  detection,  snow  area  analysis  or  water  pollution  area  detection,  in  order  to  find 
pollution  components  in  terms  of  mixture  components.  We  have  started  to  look  at  this 
using  RBF  neural  network  technology. 

6.  We  have  almost  completed  an  ad  hoc,  menu  based,  users  interface,  which  could 
facilitate  considerably  the  use  of  the  software  in  the  further  development  of  our 
methodological  research  along  the  lines  of  the  proposal. 


2 


Unfortunately,  since  the  fourth  interim  report  (April  1, 1999)  we  received  two  stop 
work  orders  for  insufficient  future  funding,  the  last  one  valid  as  of  October  first. 
Consequently,  since  then  there  was  no  capacity  for  active  software  development  and 
implementation.  So,  most  of  our  work  over  the  summer  up  to  now  was  spent  on 
studying  necessary  adaptations  of  the  software  to  various  operational  requirements, 
assisted  by  data  runs  to  suggest  their  use,  and  studying  and  assessing  recent  statistical 
and  neural  theoretical  developments  for  possible  application. 


(2)  Research  plans  for  the  remainder  of  the  period. 

During  this  period  we  received  the  disturbing  news  that  further  funding  of  this  project 
beyond  this  reporting  period  (i.e.  as  of  1  October  1999)  is  not  sure.  It  should  be 
regretted  if  the  continuation  of  this  cooperative  research  project  on  behalf  of  the 
USAGE  labs  and  divisions  in  general,  and  in  particular  of  its  CRREL-RS/GISC  staff, 
with  whom  we  had  such  good  and  mutually  stimulating  contacts  over  the  years  would 
come  to  an  abrupt  end.  The  consequence  of  this  would  be  that  the  results,  as 
mentioned  above,  as  well  as  below,  can  not  be  further  developed  and  completed,  or 
made  available  to  USAGE  and  its  clients.  This  indeed  would  be  a  waste  of  R  &  D 
capital  investment  over  the  last  four  years. 

However,  assuming  restored  funding  and  continuation  of  this  project,  we  envisage 
completing  the  rebuild  and  reengineering  design  projects  mentioned  above  and 
continuing  the  lines  as  mentioned  in  the  original  proposal. 

1.  innovative  optimization; 

2.  spectral  de-mixing; 

3.  unsupervised  change  detection. 

4.  methods  for  evaluating  classification  error  and  accuracy. 

We  shall  then  proceed  according  to  the  following  plan. 

A).  Study  innovative  improvements  in  algorithmic  petformance  and  classification 
accuracy. 

This  focuses  on  methodological  and  algorithmic  (precision,  convergence  and  speed) 
enhancement  of  the  basic  features  of  this  set  of  unsupervised  classification 
procedures.  Accuracy  evaluation  and  measurement  is  here  associated  with  the  errors 
of  misclassification,  known  in  standard  supervised  classification  methods  such  as  in 
parametric  discriminate  analysis  and  trained  artificial  neural  networks.  As  these 
methods  are  based  on,  and  presuppose  the  matching  of  image  pixels  to  previously 
known  class  or  ground  coverage  parameters,  the  concepts  of  misclassification  and 
errors  of  misclassification  can  be  immediately  defined  and  modeled  in,  for  instance, 
the  well-known  confusion  matrices. 

However,  in  non-parametric,  unsupervised  classification  these  concepts  do  not  have 
an  immediate  meaning  because,  by  definition,  no  such  prior  knowledge  exists  as  the 
whole  procedure  aims  to  detect,  define  and  generate  a  posteriori  the  system  of 
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classes  as  contained  in  the  spectral  information  of  a  particular  image  and  its 
subsequent  mapping  to  the  pixels  of  that  image. 

Hence  there  is  no  direct  analogue  here  to  the  familiar  confusion  matrix.  We  propose 
to  study  possibilities  to  develop  analogous  measures  and  concepts  in  the  context  of 
the  two  levels  that  matter  in  our  system  of  unsupervised  classification: 

(1)  the  reliability  of  the  detected  class  system  (how  stable  are  its  detected 
classes  under  pixel  sampling  and  picture  window); 

(2)  the  accuracy  of  the  pixel  class  mapping  (are  similar  pixels  mapped  to  the 
same  class?). 

A  different  problem  is  how  classes  and  pixel  mappings  in  an  image  correspond  with 
available,  usually  occasional  ground  knowledge.  This  concerns  the  validity  of  the 
obtained  set  of  classes  and  its  image  mapping.  This  we  shall  study  and  assess  the 
examination  of  classification  results  for  images  with  substantive  patches  of  quality 
ground  knowledge  with  two  major  purposes: 

-  calibration  and  tuning:  do  the  known  patches  correspond  to  discernible 
classes,  and  which  levels  of  classification  parameters  serve  to  get  the  best 
correspondence?  (at  what  levels  can  clouds  or  vegetation  be  separated  from 
water); 

-  generalization  potential:  does  the  coverage  in  a  known  patch  correspond 
with  that  of  other  patches  with  the  same  class? 

In  order  to  enable  this  type  of  study  it  was  decided  that  CRELL  RS-GISC  (Dr. 
LaPotin)  will  send  us  some  images  with  lots  of  pin-pointed  ground  knowledge  {e.g. 
oil-spill  data,  urban/rural,  coastline  areas,  snow  cover,  flood  data?),  which  are 
sufficiently  annotated  for  us  here  to  identify  known  patches  of  ground  cover  and  use 
them  for  further  quality  assessment,  validation  and  calibration. 

B).  Study  spectral  de-mixing;  within  classes  at  the  (sub-)pixel  or  level. 

In  conventional  classification  methods,  such  as  the  ISODATA  module  in  the 
ERD AS/IMAGINE  system,  pixels  are  allocated  to  a  class  just  in  terms  of  a  particular 
class  label.  In  our  system,  in  addition  to  that,  classes  are  characterized  in  terms  of  a 
univariate  distribution  along  a  linear  segment,  as  determined  by  the  first  principal 
component  of  their  singular  value  decomposition.  Pixels  can  thus  be  represented  as 
particular  distribution  values  within  that  class  (the  blending  option).  This  makes  it 
possible  to  attack  mixed  pixel  blending  as  a  mixture  problem. 

For  instance,  it  was  pointed  out  to  us  during  the  last  meeting  at  CRREL  RS/GISC  that 
certain  patches  in  the  mountain  area  were  labeled  as  belonging  to  the  same  class  as 
water  in  the  Painted  Rock  reservoir.  Obviously,  in  actual  applications  one  usually 
could  seek  to  separate  these  classes  by  choosing  a  higher,  more  discriminating  density 
value  for  class  merging.  This,  however,  would  change  the  whole  classification  pattern 
and  structure  across  the  full  image.  Instead  of  that,  in  our  case  one  could  restrict 
oneself  just  to  the  investigation  of  the  distribution  within  that  class  looking  for 
multimodality  or  other  mixing  components.  Using  statistical  demixing  techniques  one 
could  thus  decompose  (’de-mix’)  that  single  class  into  sub-classes  corresponding  to 
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’shadow’  and  regular  water.  This  approach  might  also  be  feasible  for  the  analysis  of 
snow  cover. 

Anticipated  further  work  in  this  field  is  the  development  of  non-parametric  models 
more  general  then  the  linear  singular  value  decomposition  within  classes.  We  will 
investigate  whether  recent  neural  network  theory  based  on  radial  basis  functions 
(RBFs),  and  the  use  of  spiking  neurons  can  show  the  same  promises  here  as  were 
propagated  elsewhere.  Together  with  the  use  of  evolutionary  computation  methods  to 
search  for  models  for  demixing  of  clusters  consisting  of  multiple  classes,  and  the 
usage  of  Bayesian  methods  to  exploit  the  spatial  the  spatial  structure  during  pixel 
classification.  Spatial  structure  is  exploited  by  computing  prior  probabilities  over  a 
spatial  neighborhood,  and  use  these  to  compute  posterior  pixel  classification 
probabilities. 

For  all  these  studies  we  need  images  with  varied  reliable  designated  ’true’  known 
ground  knowledge  patches.  During  our  5-7  October  meeting  at  CRREL  RS/GISC  it 
was  agreed  with  dr.  LaPotin  that  we  shall  obtain  at  least  three  images  to  serve  that 
purpose  (oil  spill,  urban/rural,  coastline  areas,  snow  cover?). 

C).  Unsupervised  change  detection 

In  later  stages  of  this  project,  assuming  sufficiently  confidence  inspiring  results  from 
this  research,  we  can  try  to  make  a  start  with  the  application  of  these  methods  and 
techniques  of  unsupervised  classification  to  the  corresponding  multi-image  problem 
of  detecting  change  areas  comparing  two  or  more  similar  coordinated  and  registered 
images,  such  as,  for  instance,  provided  by  time-sequential  oveipasses  by  LANDSAT 
TM  RS  imagery.  Results  in  this  respect  might  assist  GIS-users  in  the  area  of 
emergency  management. 
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(3)  Signiflcaat  administrative  actions  during  the  report  period 

During  the  reported  period  there  has  been  no  change  in  personnel  involved  in  the 
project. 


(4)  Other  important  information 

None 


(5)  Statement  showing  the  amount  of  unused  funds 


See  attachment. 
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Annex  to 

Fifth  Interim  Report  March  till  November  1999 

Unsupervised  Spatial  Feature  and  Change  Detection  in  RS  and  Imaging 


contract  no.  N  68171  98  C  9012 
contractor  Prof.dr  R.J.  Mokken 
ALL/CCSOM,  PSCW,  University  of  Amsterdam 


1.  Statement  shoM^ing  amount  of  unused  funds  at  the  end  of  the  covered  period 


2nd  Incrementally  Funded  Period 

November  '99-March  00 

total 

$  33,921 

3rd  Incrementally  Funded  Period 

total 

$  48,867 

April  00  -  March  01  _ 

total  unused  funds  at  end  of  covered  period  $  82,663 

2.  List  of  important  property  acquired  with  contract  funds  during  this  period 

none 
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